Papers with unseen datasets
InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Instruction tuning is emerging in NLP, but has not been explored for dialogue-related tasks. |
| Approach: | They propose an instruction tuning framework for dialogue that leverages natural language instructions with language models to induce zero-shot generalization on unseen tasks. |
| Outcome: | The proposed framework enables good zero-shot performance on unseen datasets and tasks such as dialogue evaluation and intent detection. |
ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning (2023.acl-long)
Copied to clipboard
| Challenge: | Pretraining has been shown to scale well with compute, data size and data diversity. |
| Approach: | They propose a method that provides benefits of multitask learning but leverages distributed computation . they propose 'coldfusion' can create synergistic loop where finetuned models can be "recycled" |
| Outcome: | The proposed method outperforms RoBERTa and previous multitask models on 35 datasets. |
LRQuant: Learnable and Robust Post-Training Quantization for Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for post-training quantization (PTQ) are limited by the complexity of the quantization parameter and performance degradations when tested on unseen datasets. |
| Approach: | They propose a learnable smooth-based PTQ framework that allows for rapid adaptation during testing. |
| Outcome: | The proposed framework improves performance on unseen datasets and reduces memory constraints. |
GenTKG: Generative Forecasting on Temporal Knowledge Graph with Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for temporal relational forecasting are limited and require limited training data. |
| Approach: | They propose a retrieval-augmented generation framework that uses temporal logical rule-based retrieval and parameter-efficient instruction tuning to solve temporal knowledge forecasting challenges. |
| Outcome: | The proposed framework outperforms conventional methods in the temporal knowledge graph domain with low computation resources. |
Towards Database-Free Text-to-SQL Evaluation: A Graph-Based Metric for Functional Correctness (2025.coling-main)
Copied to clipboard
Yi Zhan, Longjie Cui, Han Weng, Guifeng Wang, Yu Tian, Boyi Liu, Yingxiang Yang, Xiaoming Yin, Jiajun Xie, Yang Sun
| Challenge: | Existing metrics for evaluating functional correctness of SQL queries are prone to false positives due to inadequately prepared test databases. |
| Approach: | They propose a graph-based metric that uses a relational operator tree to extract rich semantic information from the logical execution plan of SQL queries and embed it into a diagram. |
| Outcome: | The proposed method eliminates the need for extensive test database preparation and performs graph matching on unseen SQL queries. |
IterCQR: Iterative Conversational Query Reformulation with Retrieval Guidance (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for conversational query reformulation depend on human annotations. |
| Approach: | They propose a method that reformulates context-dependent conversational queries without relying on human rewrites. |
| Outcome: | The proposed method shows state-of-the-art performance on two widely-used datasets. |
ReinforceBug: A Framework to Generate Adversarial Textual Examples (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that ML Models are sensitive to Adversarial Examples (AEs) AEs are generated by perturbingining examples that preserve the intrinsic utility of the ML solutions but influence the classifier's predictions between original and modified inputs. |
| Approach: | They propose a reinforcement learning framework that learns a policy that is transferable on unseen datasets and generates utility-preserving and transferable AEs. |
| Outcome: | The proposed framework is 10% more successful than the state-of-the-art attack TextFooler. |
MoDE-CoTD: Chain-of-Thought Distillation for Complex Reasoning Tasks with Mixture of Decoupled LoRA-Experts (2024.lrec-main)
Copied to clipboard
| Challenge: | Current Chain-of-thought Distillation methods hinder CoT reasoning performance . student models are separately distilled from specific reasoning tasks . parameter update of student models severely harms CoT ability on unseen reasoning tasks. |
| Approach: | They propose a method which distills Chain-of-thought reasoning ability of large language models to much smaller student models. |
| Outcome: | The proposed method improves the reasoning ability of large language models on 14 datasets. |
Towards Event Extraction with Massive Types: LLM-based Collaborative Annotation and Partitioning Extraction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Event Extraction (EE) is a long-standing target, but lacks an efficient and effective annotation framework to construct the corresponding datasets. |
| Approach: | They propose an LLM-based collaborative annotation framework that refines annotations of triggers from distant supervision and carries out argument annotation. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on the largest EE dataset to date . it achieves the F1 scores of 90% and 85.3% on the human-annotated test set . |